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1  TECHNICAL  PROGRESS/ SIGNIFICANT  ACCOMPLISHMENTS 
AND  EVENTS  OVER  THE  CURRENT  REPORTING  PERIOD 
(PLUS  HISTORIC  PROGRESS) 

April  201 1 

•  Work  plan  shifted  to  focus  on  final  reporting  requirement  according  to  updated  contract 

•  Scheduled  full  team  meeting  with  all  collaborators  for  May  19-20  in  Los  Angeles 

•  Additional  background  research  on  language,  culture  2:  Colombian  Spanish 

o  Literature  review  by  Dr.  Suzanne  Wertheim  resulted  in  updates  to  cultural  points  of 
difference  documentation 

o  Extension  of  the  coverage  of  the  microsocial  model  by  Dr.  Jerry  Hobbs 

March  201 1 

•  Work  proceeded  on  tasks  from  the  new  SOW 

•  Background  research  on  language,  culture  2:  Colombian  Spanish 

o  Data  development  by  Dr.  Michael  Agar  resulted  in  written  summary  of  cultural  points 
of  difference  for  Colombia  and  Afghanistan 
o  Extension  of  the  coverage  of  the  microsocial  model  by  Dr.  Jerry  Hobbs 
o  Preliminary  design  of  the  user  experience  for  October  2011  prototype  with  key 
“value-added”  capabilities 

February  2011 

•  Internal  kick-off  completed  February  3-4  in  Los  Angeles,  with  all  team  members 

•  Presented  work  on  this  project  in  talks  at  HSCB  Focus  201 1  conference,  February  7-10 

o  Lewis  Johnson:  Model-Based  Cultural  Competence  Training 
o  Alicia  Sagae:  Usability  in  MMT  for  Up-To-Date  Cultural  Competence  Training 
o  LeeEllen  Friedland:  Front-End  Validation  of  Data  for  Micro-Social  HSCB  Models 

•  Current  work  is  pending  approval  of  the  new  SOW 


January  201 1 

•  Created  Development  Scenario  for  project  year  two,  CDRL  A001  (update) 

o  Submitted  with  this  report 

•  Updated  the  project  Statement  of  Work  to  reflect  expected  resources  for  March-October  201 1 

o  Submitted  separately  by  email  to  Dr.  Ivy  Estabrooke,  cc:  Gary  Kollmorgen  and  Clark 
Phillips 

o  Includes  updated  spend  plan,  also  submitted  with  the  SOW  by  email 
December  2010 
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•  Caught  up  invoicing  against  funding  increment  received  on  November  30,  2010 

•  Started  work  on  updated  spend  plan  and  deliverables  schedule  to  accommodate  2010  funding 
gap 

November  2010 

•  Project  activity  has  been  on  hold  pending  funding  for  project  year  two 

•  On  November  30,  received  new  funding  increment  covering  at-risk  work  executed  in 
September  and  October  2010 

October  2010 

•  Prepared  submission  for  exhibition  booth  at  HSCB  Focus  2011  conference 

•  Other  project  activity  is  on  hold  pending  funding  for  project  year  two 

•  Please  note  the  updated  contact  information  for  our  financial  team:  Izett  Barnett,  Director  of 
Finance  (ibamett@alelo.com) 

September  2010 

•  Participated  in  ONR  Capabilities  Open  House,  September  20-21,  2010 

•  Other  project  activity  is  on  hold  pending  funding  for  project  year  two 


August  2010 


•  Completed  year  1  Technical  Performance  Evaluation  (TPE)  in  Arlington,  VA 

•  Started  preparations  for  ONR  Capabilities  Open  House  in  September  2010 


July  2010 


•  Completed  year  1  evaluation  and  submitted  evaluation  report  (CDRL  A005) 

•  Submitted  update  to  TCA  design  doc  (CDRL  A003) 

•  Submitted  update  to  culture  model  document  (CDRL  A002) 


June  2010 


•  Implemented  test  cases  that  can  be  run  in  the  framework  completed  last  month 

•  Continuing  development  of  behavior  rules  to  pass  these  test  cases 

•  Continuing  development  work  for  Culture  Model  and  TCA  Design 

•  Outlined  year  1  evaluation  report  and  assigned  writing  tasks 


May  2010 


•  Updated  Model  Design  Document  and  Evaluation  Design  Document  from  April  2010 


Copyright  ©  2011  Alelo  TLT,  LLC 
Information  not  to  be  disclosed  other  than  by  intended  recipient 


2 


Alelo  TLT,  LLC 
Project:  HSCB  CultureCom 


Contract:  N00014-09-C-0613 
Monthly  Report  for  April  2011 


deliverables,  according  to  follow-up  discussion  with  Dr.  Ivy  Estabrooke  (attached) 

•  Introduced  and  trained  new  student  intern  who  will  work  on  the  evaluation  team  between 
May  and  August,  2010 

•  Implemented  testing  framework  to  support  evaluations  outlined  in  CDRL  A004 

April  2010 

•  Prepared  and  submitted  updates  to  the  Microsocial  Model  Document  (CDRL  A002)  and 
TCA  Design  Document  (CDRL  A003)  on  schedule,  on  April  30,  2010 

•  Prepared  and  submitted  Year  1  Evaluation  Criteria  (CDRL  A004),  on  schedule,  on  April  30, 
2010 

•  Annotated  test  dialogs  with  gold-standard  logical  axioms  for  year  1  evaluation 

•  Updated  the  publication  inventory  (submitted  with  this  report) 

March  2010 

•  Continued  data  development  on  interview  material  from  SME2 

•  Identified  test  dialogs  for  year  1  evaluations  and  defined  test  cases  over  these  dialogs 

•  Started  implementation  of  the  test  cases  and  preparation  of  the  year  1  evaluation  criteria 
document 

•  Continued  implementation  work  for  TCAs  and  preparation  for  the  updated  TCA  document 
due  in  April 

•  Continued  extension  of  the  microsocial  model  and  preparation  for  the  updated  model  design 
document  due  in  April 

•  Inventoried  publications  that  have  been  produced  on  this  project  so  far  (submitted  with  this 
report) 

Lebruary  2010 

•  Prepared  for  and  presented  6-month  review  with  ONR  in  San  Lrancisco,  CA  on  Lebruary  9th 

•  Prepared  a  revised  budget  with  justification  and  submitted  to  ONR  on  Lebruary  18th 

•  Prepared  an  updated  revised  budget  with  new  LY2010  spend  target  of  $250,000;  budget  is 
now  under  review  by  financial  team  and  will  be  submitted  this  week  (before  3/12/2010) 

•  Data  collection:  performed  data  development  on  interview  material  from  SME  2 

January  2010 

•  Data  collection:  conducted  initial  interview  with  SME  2,  screened  candidates  for  SME  3 

•  Started  implementation  work  based  on  the  TCA  design  document  delivered  in  December 

•  Prepared  materials  for  6-month  review  meeting  in  San  Francisco,  CA  in  early  February  2010 

December  2009 

•  Delivered  first  draft  of  culture  model  document  (CDRL  A002)  on  December  24,  2009 
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•  Delivered  first  draft  of  TCA  design  document  (CDRL  A003)  on  December  24,  2009 

•  Started  scheduling  for  6-month  review  meeting  in  San  Francisco,  CA  in  early  February  2010 

November  2009 

•  Reviewed  Data  Collection  process  with  Alelo  Culture  Team  and  Dr.  Agar;  revised  Data 
Requirements  Document  (DRD);  extracted  sample  dialogs  from  dry-run  data  collection 
executed  in  October;  created  data-collection  schedule  for  December  2009-March  2010: 

o  December  2009:  Polish  Interview  Protocol  Document  (IPD)  with  input  from  Mike 
Agar;  conduct  follow-up  interview  with  SME  1  (dry-run  SME);  complete  the 
technical  design  document  and  culture  model;  schedule  phone  conference  with  ONR 
on  cross-cultural  competencies. 

o  January  2010:  Extract  dialogs  from  follow-up  interview  with  SME  1;  choose  2 
additional  SMEs  and  schedule  initial  and  follow-up  interviews. 

o  February-March  2010:  conduct  interviews  and  extract  20-30  dialogs  from  SME  2, 
SME  3;  identify  development  and  test  dialogs  for  year  1  evaluations;  update  dialogue 
system  and  design  documentation  based  on  this  data. 

•  Created  first  draft  of  culture  model  document  for  deliverable  in  December,  2009 

•  Implemented  prototype  of  text-based  conversational  agent  (TCA)  dialog  loop  and  created 
first  draft  of  design  document  for  deliverable  in  December,  2009 

•  Prepared  article  for  Scientist  spotlight  in  January  HSCB  Newsletter 

October  2009 

•  Held  quarterly  in-person  meeting  with  all  subcontractors  and  team  members  on  October,  27 
2009  in  Los  Angeles 

•  Completed  in-house  review/collection  of  cultural  data  resources  for  Afghanistan  with  their 
IRB-relevant  information 

•  Created  a  Data  Requirements  Document  (DRD)  to  describe  the  needs  for  additional  data 
collection 

•  Created  a  draft  Interview  Protocol  Document  (IPD)  that  defines  procedures  for  collecting 
data  described  in  the  DRD  in  interviews  with  Subject-matter  experts 

•  Executed  a  dry-run  of  the  IPD  with  Alelo  consultants  serving  as  native- speaker  language 
experts  in  Pashto 

•  Continued  work  on  alignment  between  RACCA  WG  cultural  learning  objectives  and  project 
objectives 

September  2009 

•  Held  kick-off  meeting  at  ONR  on  8  September,  2009 

•  Started  in-house  review/collection  of  cultural  data  resources  for  Afghanistan  with  their  IRB- 
relevant  information 

•  Initiated  negotiation  of  an  appointment  to  observe  a  lane  training  session  at  29  Palms  in  order 
to  collect  additional  first-person  accounts  and  sample  dialogs  from  human  role -players 
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•  Received  documentation  of  the  RACCA  WG  cultural  learning  objectives  from  Dr.  Allison 
Abbe  and  started  work  on  alignment  between  these  objectives  and  the  objectives  for  the 
CultureCom  project 

•  Continued  to  perform  background  cultural  research  and  work  on  the  first  draft  of  axioms  and 
concepts  for  the  culture  model  that  supports  our  Year  1  Development  Scenario 

August  2009 

•  Completed  contract  negotiations  with  project  start  date  31  July,  2009 

•  Selected  geographic  area  of  focus  and  tasks  of  focus  for  year  1 

•  Created  and  submitted  Year  1  Development  Scenario  (CDRL  A001) 

•  Prepared  materials  for  kick-off  meeting  08  September,  2009 

2  OUTSTANDING/ANTICIPATED  ISSUES  AND  SOLUTIONS  TO 
THOSE  ISSUES 

April  201 1 

•  Subcontract  renewal  with  the  University  of  Southern  California  has  been  severely  delayed 
due  to  difficulty  in  assessing  the  total  contract  amount 

•  Most  recent  discussions  with  program  leadership  resulted  in  a  decision  on  the  current  funding 
increment,  which  should  help  resolve  this  issue 

•  Back-invoicing  for  work  performed  by  subcontractors  between  January  201 1-May  201 1  is 
expected  to  occur  soon,  which  will  result  in  a  jump  in  invoiced  costs  to  catch  up  with  work 
that  was  performed  in  that  period 

March  2011 

•  Current  funding  expended,  accounting  for  delayed  invoices  from  subcontractors  that  are 
expected  to  arrive  in  April 

•  Project  was  instructed  29  March  2011  to  stop  work  on  current  SOW  and  prepare  a  final 
report 

February  2011 

•  Current  funding  ends  in  March 

•  Continued  work  on  the  project  is  pending  approval  of  the  updated  SOW 

o  Along  with  new  funding  increment  to  support  the  work  described  in  SOW 

January  2011 

•  Continued  work  on  the  project  is  pending  approval  of  the  updated  SOW  and  receipt  of  a  new 
funding  increment  to  support  the  work  described  there 
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December  2010 

•  The  project  schedule  is  likely  to  be  delayed  by  90-120  days  total  in  the  revised  project  plan, 
which  accounts  for  work  stoppage  due  to  funding  gaps 

November  2010 

•  The  project  schedule  is  likely  to  be  delayed  by  90-120  days  total  in  the  revised  project  plan, 
which  accounts  for  work  stoppage  due  to  funding  gaps 

October  2010 

•  The  project  schedule  is  likely  to  be  delayed  an  additional  30  days  (60-90  days  total)  once 
activity  resumes,  pending  year  two  funding 

September  2010 

•  The  project  schedule  is  likely  to  be  delayed  an  additional  30  days  (60-90  days  total)  once 
activity  resumes,  pending  year  two  funding 

August  2010 

•  The  schedule  for  project  year  two  will  be  delayed  by  30-60  days,  due  to  the  following  issues 


o  Insufficient  time  was  planned  in  the  August  schedule  to  accommodate  year-end  travel 
and  reporting  for  year  one  along  with  year-two  kickoff  activities;  however,  the  year- 
end  activities  were  worth  the  investment  in  time  since  they  resulted  in  helpful 
communication  with  the  sponsors  that  we  will  apply  going  forward 
o  The  current  funding  increment  has  ended  (at  time  of  writing  we  have  approximately 
two  weeks  left,  depending  on  level  of  effort) 
o  We  are  moving  our  staff  to  other  projects  for  the  time  being,  but  we  plan  to  be  ready 
to  start  CultureCom  work  again  as  soon  as  funding  permits 


July  2010 

•  No  known  issues 
June  2010 

•  No  known  issues 
May  2010 

•  No  known  issues 
April  2010 
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•  No  known  issues 
March  2010 

•  No  known  issues 
February  2010 

•  No  known  issues 
January  2010 

•  No  known  issues 
December  2009 

•  No  known  issues 
November  2009 

•  We  have  not  yet  received  invoices  from  the  University  of  Southern  California,  a 
subcontractor  on  this  project.  As  a  result  the  hours  worked  by  USC  have  not  yet  been 
charged  to  the  spend  plan.  However  we  do  expect  to  receive  an  invoice  in  the  month  of 
December  which  will  correct  our  spending  with  respect  to  the  plan. 

October  2009 

•  Alelo  resolved  the  spend  plan  issue  from  last  month  by  completing  a  revised  spend  plan; 
submitted  to  ONR  and  SA,  Inc.  by  email  on  26  October,  2009 

•  We  have  not  yet  scheduled  a  phone  conference  with  ONR  on  the  topic  of  cultural  learning 
objectives;  We  engaged  or  collaborators  on  this  project,  both  subcontractors  (Dr.  Michael 
Agar)  and  other  colleagues  (Dr.  LeeEllen  Friedland)  in  discussions  of  this  topic  and  we  are 
ready  to  schedule  the  conference  for  the  month  of  November  at  ONR’s  convenience 

September  2009 

•  Spend  plan  should  be  revised  and  we  have  already  started  this  work;  original  spend  plan 
assumed  a  constant  rate  of  expenditures,  when  in  fact  the  costs  during  the  initial  months  will 
be  lower  while  costs  during  software  development  and  integration  months  will  be  higher;  we 
are  working  on  a  revised  spend  plan  that  reflects  this  change  in  our  method  of  estimation  as 
well  as  new  hires  that  we  plan  to  take  on  for  the  upcoming  software  development  phase 

August  2009 

•  No  known  issues 
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3  FUTURE  SIGNIFICANT  EVENTS 

April  2011 

•  Full  team  meeting  to  conclude  background  research  for  second  culture  of  interest  scheduled 
for  19-20  May  in  Los  Angeles 

•  Start  of  SME  interviews  for  second  culture  of  interest,  to  support  comparisons  in  final  report, 
beginning  early  June 

March  2011 

•  Pending  phone  conference  with  sponsors,  new  research  will  stop  on  the  project  and  effort 
will  shift  to  producing  a  project  final  report,  addressing  future  directions: 

o  How  to  expand  to  a  complete  Pashto  (Afghanistan)  training  system 
o  Time  and  resources  required  to  move  from  English  to  Pashto  for  the  Aftghanistan 
culture  system 

o  Time  and  resources  required  move  from  Afghanistan  to  a  2nd  culture  of  interest  at 
lowest  cost  possible  and  at  lowest  time  possible 


February  2011 

•  Pending  approval  of  the  new  SOW: 

o  Recruitment  of  Subject  Matter  Experts  for  ethnographic  interviews 
o  Technical  review  of  results  from  year  one  and  start  of  design  work  for  year  two 
prototype 

o  Background  research  and  information  gathering  by  the  culture  modeling  team 
January  2011 

•  Internal  kick-off  scheduled  for  February  3-4  in  Los  Angeles,  with  all  team  members 

•  Project-related  presentations  will  be  given  at  HSCB  Focus  2011  Conference  February  7-10 

•  Future  work  is  pending  approval  of  the  new  SOW 

o  Recruitment  of  Subject  Matter  Experts  for  ethnographic  interviews 
o  Technical  review  of  results  from  year  one  and  start  of  design  work  for  year  two 
prototype 

o  Background  research  and  information  gathering  by  the  culture  modeling  team 
December  2010 

•  Project-related  presentations  will  be  given  at  HSCB  Focus  2011  Conference  in  February 

•  Updated  project  plan  is  still  under  development  to  identify  year  two  activities  that  may  be 
funded  with  current  increment 

•  Project  year  two  internal  kick-off  is  likely  to  occur  in  January  201 1 
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November  2010 

•  Updated  project  plan  will  be  produced  in  December  to  identify  year  two  activities  that  may 
be  funded  with  current  increment 

•  Project  year  two  internal  kick-off  is  likely  to  occur  in  January  201 1 
October  2010 

•  Project  year  two  internal  kick-off  may  occur  in  November  or  December  2010 
September  2010 

•  Project  year  two  internal  kick-off  may  occur  in  November  2010 

August  2010 

•  Dry-run  on  September  10th  for  the  ONR  Capabilities  Open  House 

•  Pending  dry-run  results,  ONR  COH  on  September  20th-21st 


July  2010 


•  Technical  Performance  Evaluation  (TPE)  will  be  held  at  MITRE  on  August  11,  2010 


June  2010 


•  Remaining  milestones  (internal  and  external)  for  July: 

o  Populate  a  series  of  test  cases  from  the  Year  1  evaluation  dialogs  (Completed) 
o  Populate  TCA  rule  instances  based  on  these  test  cases  (7/8) 

o  Calculate  metrics  from  the  evaluation  criteria  document  over  these  test  cases  (7/16) 
o  Generate  and  submit  report  documents  with  evaluation  results  (7/31) 


May  2010 


•  Next  external  milestone  is  evaluation  results  with  a  year-1  final  report  in  July,  2010 

•  Internal  milestones  for  June  and  July: 

o  Populate  a  series  of  test  cases  from  the  Year  1  evaluation  dialogs  (6/24) 
o  Populate  TCA  rule  instances  based  on  these  test  cases  (7/8) 

o  Calculate  metrics  from  the  evaluation  criteria  document  over  these  test  cases  (7/16) 
o  Generate  and  submit  report  documents  with  evaluation  results  (7/31) 


April  2010 


•  A  student  intern  will  join  our  team  starting  in  May,  2010.  His  work  will  focus  on 

implementation  of  the  evaluation  dialogs  using  TCA  behavior  rules,  and  on  supporting  our 
year  1  evaluation  experiments  and  written  reports. 
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March  2010 


•  Next  milestone  is  evaluation  criteria  with  updated  TCA  design  and  culture  model,  due  April 
30,  2010 

February  2010 

•  Next  milestone  is  evaluation  criteria  with  updated  TCA  design  and  culture  model,  due  April 
30,  2010 

January  2010 

•  6-month  review  meeting  scheduled  for  Tuesday,  February  8th  in  San  Francisco,  CA. 

December  2009 

•  We  have  been  in  contact  with  Dr.  Allison  Abbe  regarding  our  follow-up  phone  conference  on 
cross-cultural  learning  objectives  and  we  hope  to  have  an  opportunity  to  talk  with  her  this 
month  (January  2010). 

November  2009 

•  Drafts  of  the  Culture  Model  and  the  TCA  Design  document  are  due  to  be  submitted  to  ONR 
in  December,  2009 

•  Please  note:  the  Alelo  address  and  phone  number  have  changed.  Updated  contact 
information  is  given  on  the  cover  page  of  this  Monthly  Report,  and  repeated  here  for 
convenience: 

Alelo 

12910  Culver  Blvd.  Suite  J 

Los  Angeles,  CA,  90066 

Telephone  (for  Alicia  Sagae):  310-574-7528 

October  2009 

•  Current  data  collection  efforts  using  the  DRD  and  IPD  described  in  Section  1  should  result  in 
additional  example  dialogs  becoming  available 

•  These  dialogs  will  be  used  to  focus  our  current  development  work  on  the  Culture  Model  and 
the  Text-based  Communicative  Agents  Design  Document  (TCA  Design);  drafts  of  these 
documents  are  due  to  be  submitted  to  ONR  in  December,  2009 

September  2009 

•  A  visit  to  the  live  training  simulations  at  29  Palms  will  hopefully  take  place  in  October  or 
November  2009. 

•  An  internal  project  meeting  with  all  collaborators  is  scheduled  for  October  26,  2009. 
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August  2009 

•  The  project  kick-off  meeting  at  Strategic  Analysis  scheduled  for  08  September,  2009. 


4  TOTAL  MONTHLY  BILLED  EXPENDITURE  DOLLAR 
AMOUNTS  AND  INCLUDE  COPIES  OF  “ACTUAL  INVOICES” 
SENT  FOR  PAYMENT  WITH  THIS  REPORT. 


Expenditures  for  the  month  of  March  201 1 
FY  2010:  $8,157* 

*  Subcontractor  expenditures  during  March  estimated  at  $9,700  but  have  not  been  invoiced  yet 

Expenditures  for  the  month  of  March  201 1 
FY  2010:  $11,605* 

*  Subcontractor  expenditures  during  March  estimated  at  $9,700  but  have  not  been  invoiced  yet 

Expenditures  for  the  month  of  February  201 1 
FY  2010:  $15,994.47 

Expenditures  for  the  month  of  January  201 1  (adjusted  to  invoice  amount) 

FY  2010:  $6,565 

Expenditures  for  the  month  of  December  2010 
FY  2010:  $1,356 

Expenditures  for  the  month  of  November  2010 

Work  at  risk  (self-funded,  will  invoice  if  additional  funding  becomes  available):  $21 1 
Expenditures  for  the  month  of  October  2010 

Work  at  risk  (self-funded,  will  invoice  if  additional  funding  becomes  available):  $636 
Expenditures  for  the  month  of  September  2010 
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Work  at  risk  (self-funded,  will  invoice  if  additional  funding  becomes  available):  $12,667 

Expenditures  for  the  month  of  August  2010 
FY2010  (ACRN  AB):  $38,765 
FY2009  (ACRN  AA):  $70,449 

Expenditures  for  the  month  of  July  2010 
FY2010:  $32,935 

*This  invoice  does  not  include  expenses  for  June  or  July  for  our  subcontractor,  the  University  of  Southern 
California.  We  continue  to  pursue  timely  invoicing  from  USC.  Our  current  estimate  of  the  un-invoiced 
cost  is  $17,024. 

Expenditures  for  the  month  of  June  2010 
FY2010:  $31,851 

*This  invoice  does  not  include  expenses  for  our  subcontractor,  the  University  of  Southern  California. 
July  expenditures  for  USC  will  be  combined  with  August,  2010  expenditures  due  to  a  delay  in  USC  posting 
invoices  to  Alelo. 

Expenditures  for  the  month  of  May  2010 
FY2010:  $21,340 

Expenditures  for  the  month  of  April  2010 
FY2010:  $48,011 

Expenditures  for  the  month  of  March  2010 
FY2010:  $67,143 

Expenditures  for  the  month  of  February  2010 
FY09:  $71,588.99 

Expenditures  for  the  month  of  January  2010  (Note:  total  billed  amount  is  under  budget  at  this  time  due 
to  delay  in  receiving  invoices  from  all  subcontractors) 

FY09:  $79,759.90  (January  and  December) 

Expenditures  for  the  month  of  December  2009 


FY09:  $0 


Copyright  ©  2011  Alelo  TLT,  LLC 
Information  not  to  be  disclosed  other  than  by  intended  recipient 


12 


Alelo  TLT,  LLC 
Project:  HSCB  CultureCom 


Contract:  N00014-09-C-0613 
Monthly  Report  for  April  2011 


Expenditures  for  the  month  of  November  2009  (Note:  total  billed  amount  is  under  budget  at  this  time 
due  to  delay  in  receiving  invoices  from  all  subcontractors) 

FY09:  $15,791 

Expenditures  for  the  month  of  October  2009 
FY09:  $9,531 

Expenditures  for  the  month  of  September  2009 
FY09:  $12,020 

Expenditures  for  the  month  of  August  2009 
FY09:  $4,446 
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